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ABSTRACT 

This paper reports a comprehensive "meta-review" and 
synthesis of research on variables related to learning, including 
both cognitive and affective schooling outcomes. A conceptual 
framework was developed, encompassing 228. items related to school 
learning, organized "a priori" into 30 scales within 6 categories 
ordered roughly from more distal to more proximal factors: (1) state 
and district variables; (2) out of school contextual variables; (3) 
school-level variables; (4) student variables; (5) program design 
variables; and (6) implementation, classroom instruction, and climate 
variables. Search and selection procedures yielded 179 selected 
handbook and annual review chapters, commissioned papers, and other 
authoritative reviews. Content analysis yielded over 3,700 ratings of 
the strength of influence of the variables on learning. They confirm 
the primacy of student, classroom, home, and community influences on 
learning relative to more distal policy variables such as state and 
district characteristics. They also highlight the importance of 
metacognition, classroom management, quantity of instruction, 
classroom interactions, classroom climate, and the peer group. 
(Contains 204 references.) (Author/DB) 
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Abstract 



This paper reports a comprehensive "meta-review" and synthesis of 
research on variables related to learning, including both cognitive and affective 
schooling outcomes. A conceptual framework was developed, encompassing 228 
items related to school learning, organized a prinri into 30 scales within six 
categories. Search and selection procedures yielded 179 selected handbook and 
annual review chapters, commissioned papers, and other authoritative reviews. 
Content analysis yielded over 3,700 ratings of the strength of influence of the 
variables on learning. They confirm the primacy of student, classroom, home, 
and community influences on learning relative to more distal policy variables 
such as state and district characteristics. They also highlight the importance 
of metacognition, classroom management, quantity of instruction, classroom 
interactions, classroom climate, and the peer group. 
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Variables Important to Learning: 
A Meta-Review of Reviews of 
the Research Literature 



Educational research has identified a large number of variables related to 
school learning. Indeed, such a multiplicity of distinct influences on 
achievement have been found that educators may be perplexed as to which 
items are most important. Educational researchers, policy makers, and 
practitioners all require clearer guidance concerning the relative importance of 
different learning influences and the particular variables most likely to 
maximize school learning. To address this need, a comprehensive review and 
synthesis of handbooks, review annuals, and other highly synthetic prior 
reviews was undertaken. Its purpose was to characterize the most 
authoritative scholarly opinion about ways to optimize educational outcomes 
across a range of educational conditions and settings. This research synthesis 
is distinguished by its comprehensiveness, its orientation toward practical 
school improvement strategies, and its focus on comparing the relative 
contributions of different items to learning. To organize the synthesis, a 
conceptual framework was developed which draws heavily upon major 
theoretical models of school learning. Before turning to this framework, the 
evolution of these earlier theoretical models is briefly described. 

Evolution of Models of School Learning 

J. B. Carroll (1963) introduced educational researchers to models of school 
learning in his Teachers College Record aptly entitled article, "A Model of 
School Learning." In his model, he put forth six constructs: aptitude, ability 

to comprehend instruction, perseverance, clarity of instruction, matching the 
task to student characteristics, and opportunity to learn. These constructs, 
which succinctly capture the psychological influences on school learning, 
became a point of departure for other models to follow. The 1960s and 1970s 
were marked by the introduction < of several additional important models of 
learning, including those of Bruner (1966), Bloom (1976), Harnischfeger and 
Wiley (1976), Glaser (1976), and Bennett (1978). 
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All of these models recognize the primary importance of student ability, 
and included constructs such as aptitude, prior knowledge, verbal IQ, and pupil 
background. Most of them also address the importance of motivation, by 
employing such constructs as perseverance, self concept of the learner, and 
attitude toward school subject matter. This acknowledgement of individual 
difference variables among learners stood in contrast to more narrowly 
psychological studies of influences on learning, which generally treated 
individual differences as a source of error, focusing instead on 
instructional-treatment variables (Hilgard, 1964). 

In addition to student variables, each of the models of school learning 
noted above also gave salience to constructs developed from studies of 
classroom instruction. These constructs varied in generality, some being as 
broad as "instructional events" or "clarity of instruction," and others as narrow 
as "use of cues* or "feedback and correctives." 

Although later models brought some refinement in the ways in which 
individual difference variables and instructional variables were defined and the 
ways in which they were related to one another, the primary contributions of 
more recent models have been in extending the range of influences considered. 
Haertel, ^^alberg, and Weinstein (1983), for example, identified nine theoretical 
constructs that exhibit consistent causal influences on academic learning: 
student age or developmental level, ability (including prior achievement), 
motivation, amount or quantity of instruction, quality of instruction, 
psychological environment of the classroom, influence of the home, influence 
of the peer group outside of school, and exposure to mass media. They 
showed that previous models of school learning neglected extramural and 
social-psychological influences. 

The evolution of models of school learning was further advanced with the 
introduction of models of adaptive instruction (Wang & Lindvall, 1984; Wang & 
Walberg, 1985). School-based implementation of models of adaptive instruction 
are designed to help schools create learning environments that maximize each 
student's opportunities for success in school. It paid particular attention to 
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new variables associated with instructional delivery systems, program design, 
and implementation. It attended in particular to those features that Glaser 
(1982) referred to as the "large practical variables," and included efficient 
allocation and use of teacher and student time, a practical classroom 
management system, systematic teacher feedback and reinforcement of student 
learning behavior and progress, instructional interactions based on the 
diagnosed learning needs of individual students, and flexible administrative and 
organizational patterns responsive to program implementation and staffing 
needs. 

Another contribution to models of school learning came from sociologists 
concerned with the identification of effective schools. Ronald Edmonds (1979) 
is most strongly associated with this identification of variables associated with 
exceptionally effective schools, especially for the urban poor. Significant 

contributions to effective schools models were also made by Brookover (1979), 
Brookover and Lezotte (1979), and Rutter, Maughan, Mortimore, Ouston, and 
Smith (1979). Illustrations of the types of variables characterizing effective 
schools include degree of curriculum articulation and organization, schoolwide 
staff development, parental involvement and support, schoolwide recognition of 
academic success, maximized learning time, district support, clear goals and 
high expectations, orderly and disciplined school environment, and leadership of 
principal characterized by attention to quality of instruction (Purkey & Smith, 
1983). 

These various models of school learning all contribute a variety of items, 
or variables, that may be useful to educational practitioners. Individual 

researchers may focus their work on particular variables or constructs, but the 
purpose of this synthesis was to try to provide a synoptic view of the entire 
panoply of variables. 



Methods and Procedures 

The first step in developing the meta*review described in this paper was 
to delineate a comprehensive set of variables organized into an inclusive 
conceptual framework.. Next, a corpus of over ISO books, book chapters. 




4 



reports, and other sources was identified. The 228 items in the conceptual 
framework were listed on a detailed, fifteen-page coding form, and each of the 
sources was then coded using that form. In all, over 2,500 pages of coding 
forms were completed. Each citation or discussion of an item influencing 
learning outcomes was coded by page number, together with a notation of the 
reported strength of its influence on learning. These detailed text citations by 
page number have been placed in an archive.^ 

The detailed ratings were then recoded onto a set of summary forms, one 
for each chapter or other source, which gave overall ratings of strength of 
influence for each of the items discussed in that source. These summary 
ratings were entered into machine-readable files and analyzed to determine the 
emergent consensus on which items exert the most powerful influence on 
learning outcomes. The initial coding tabulated well over 10,000 separate 
statements in the research literature concerning of the strength of association 
between one of the 228 items and learning outcomes. These were reduced to 
over 3,700 summary ratings, which were then keyed and analyzed.’ 

Before describing the data analyses and the findings of this study, the 
development of the conceptual framework, selection of the corpus of studies, 
and coding procedures are briefly described. 

Conceptual framework for items related to learning 

The identification of a comprehensive set of items began with a close 
examination of the models of school learning described above, as well as 
selected sources, including Brophy (1986), Keogh, Major-Kingsley, 
Omori-Gordon, and Reid (1982), Wang and Walberg (1985), and Wittrock (1986). 
Potential variables were written on separate index cards, then consolidated and 
organized into a preliminary version of the final coding scheme. This draft 
coding scheme was sent to members of the Scientific Advisory Panel of the 
Center for Research in Human Development and Education at Temple 
University^. Based on detailed commentaries received from the Panel members, 
the framework was revised to include four additional items, and to improve its 
organization. 
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The final framework organized the 228 items related to learning into 30 g. 
priori scales within six broad categories. The six categories were ordered 
roughly from more distal to more proximal factors. Brief descriptions of the 
categories are presented in Table 1, together with illustrative items from each 
scale. 

Selection of a corpus of studies 

A vast research literature addresses one or more of the potential learning 
influences represented by the conceptual framework, and it clearly would not 
be possible to examine all of the thousands of original studies relevant to a 
synthesis of this scope. Indeed, even the literature of review articles is 
massive. For this reason we focused on authoritative reviews and handbook 
chapters, especially those sponsored by the American Educational Research 
Association and other organizations, and selected additional syntheses in 
government documents and other sources. A preliminary list of sources was 
reviewed by the Scientific Advisory Panel, and revised following their 
recommendations. Following this review, the sources chosen included chapters 
from the past decade or more of the Review of Research in Education, the 
Annual Review of Psychology, and the Annual Review of Sociology, as well as 
the Handbook of Research on Teaching (Wittrock, 1986), Designs for 
Compensatory Education (Williams, Richmond, & Mason, 1986), more specialized 
handbooks, and a small number of journal articles chosen to assure coverage of 
all of the areas addressed in the comprehensive framework. Initially, over 200 
articles, chapters, and other sources were identified. All of these sources 
were read, but some were excluded from the final corpus because they failed 
to address K-12 instruction in regular school settings, because they addressed 
exceptionally narrow and atypical learning outcomes, or because they were 
relevant only to rare or special-learner populations. 

A total of 179 sources were included in the final corpus of studies (see 
Appendix for a complete bibliography). All of these were relevant to a range 
of cognitive and or affective learning outcomes for K-12 learners in formal 
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educational settings. Table 2 presents a summary by type of the source 
documents included in the final synthesis. 

Coding Procedures 

Each source document was coded initially onto a detailed rating form, 
which allowed for the recording of multiple references in a single document to 
the same item. In addition to coding references to the 228 prespecified items, 
space was provided for the coding of any additional items related to learning 
outcomes, referred to on the form as supplementary items. Brief notes were 
also recorded for most sources, including page references, comments on the 
source’s overall relevance, and any limitations on the learner populations 
and/or varieties of learning outcomes addressed. This archived documentation 
has been retained by the first author. 

Each reference to an item’s relation to learning outcomes was coded on a 
three-point scale, with T representing a weak, uncertain, or inconsistent 
relation to learning; "2" representing a moderate relation; and "3" representing 
a strong relation. Where "vote counts" or proportions of confirming studies 
were reported, a "3" indicated that more than 80 percent of the studies 
discussed had found a statistically significant association of an item to 
achievement; a "2" indicated that between 40 percent and 80 percent of the 
studies found support for the relationship; and a "1" indicated less than 40 
percent in support. Where results were summarized in terms of effect sizes, a 
code of "3" was assigned to effect sizes greater than .33, "2" to effect sizes of 
.10 to .33, and "1" for smaller effect sizes. Where correlations were reported, 
"3" was used for correlations greater than .40, "2" for correlations of .15 to 
.40, and "1" otherwise. 

In many cases, the source documents did not present quantitative indices 
like effect sizes or correlations, and so it was necessary to judge the strength 
of the evidence presented from prose descriptions of the conclusions from 
bodies of research. . In these cases, the strength of the evidence presented was 
Judged weak, moderate, or strong, and coded accordingly. Even though all of 
the 228 items were defined in such a way that they were expected to relate 
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positively to learning, there were rare instances in which negative conclusions 
from the literature were reported.® 

Following the coding of all specific references by page number, ratings 
were transcribed onto a second, summary form for each source, prior to keying 
for data analysis. At this stage, a single, summary code — the average of all 
the ratings for each source document — was recorded indicating the strength 
of association for each item discussed in the source, according to the 
preponderance of the specific references noted.® 



After inspecting univariate frequency distributions for each of the 228 
separate items to assure that no values were out of range, the separate items 
were aggregated to the level of the thirty scales described in Table I. This 
was accomplished by taking the average of all non-missing values in a scale, 
for each source. In cases where a source document did not discuss any of the 
separate items in a scale, a missing data code was entered. In those rare 
cases where negative findings were coded, their negative signs were retained 
when averages were taken. 

In a second stage of data reduction, six additional variables were created, 
corresponding to the categories described in Table 1. The values of these 
variables for each source were weighted averages of all nonmissing scale values 
comprised by that category.^ Means, standard deviations, and alpha 
reliabilities for the six categories and thirty scales are presented rn Table 3. 
The reliabilities for documents (not raters) range from .71 to .99. All but four 
are greater than .80, and most exceed .90®. 



Table 3 also reports the number of sources that discussed items in each 
scale. Surprisingly, the frequency with which different scale items are 
discussed in the literature is not a reliable guide to their importance for 
learning outcomes. The Spearman rank correlation between frequencies and 
means across the 30 scales is only .lO. 



Data Analysis 
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Summary of Results 



Table 3 shows the importance of many distinct influences on school 
learning outcomes. Over all 30 scales, the mean rating was roughly 1.8, a 
little below the level designated "moderate relation to learning.” More 
important, however, the synthesis shows which categories, scales, and specific 
items are most strongly associated with learning outcomes. In discussing 
results by category and by scale, relevant findings concerning specific items 
will be presented to clarify or elaborate the category- and scale-level findings 
reported in Table 3. 

At the highest level of generality, this synthesis confirms the importance 
of the quality of schooling for learning outcomes. Of the six categories, the 
highest ratings overall were assigned to "Program Design Variables," followed 
by "Out of School Contextual Variables." The category reflecting the quality 
of instruction as delivered, "Classroom Instruction and Climate Variables," 
ranked third in importance, closely followed by "Student Variables." The last 
two categories, "School Level Variables" and "State and District Variables," 
received markedly lower ratings overall. This overall ranking of sources of 
influence contrasts sharply with the "conventional wisdom" since the time of 
the Equality Educational Opportunity (EEO) Survey (Coleman, et al., 1966) 

that quality of schooling has relatively little impact on schooling outcomes 
relative to out-of-school, socioeconomic variables. 

The importance of proximal psychological variables may be seen in the 
scales that obtained the highest ratings.* Those scales with mean ratings of 
2.00 or greater were (beginning with the highest): 



0 


Metacognition 


X - 2.08 


0 


Classroom Management 


X - 2.07 


0 


Quantity of Instruction 


X - 2.02 


0 


Student/Teacher Interactions: Social 


X - 2.02 


0 


Classroom Climate 


X - 2.01 


0 


Peer Group Influences 


X - 2.00 



In the remainder of this section, the categories and scales are discussed 
in turn, and those scales and items that received exceptionally high ratings are 
highlighted. The categories representing instruction as designed and 



instruction as delivered are discussed first. These are followed by 
out-of-school context and student characteristics. Finally, the more distal 
variable categories of school level variables and state and district variables are 
addressed. 

Program Design Variables 

This category includes instruction as designed, and the physical 
arrangements for its delivery, organized into three scales, as shown in Tables 
1 and 3. The scale "Demographic and Marker Variables" was rated highest of 
the three, and within this scale the most highly rated items are "size of 
instructional group (whole class, small group, or one-on-one instruction)," 
"number of classroom aides," and "resources needed." (Ratings for these items 
ranged from 1.95 to 2.00.) Thus, the most important aspect of program design 
appears to be the intensity of educational services provided to each learner. 
More aides, smaller groups, or increased material resources are associated with 
significantly higher learning outcomes. 

"Curriculum and Instructional Variables" includes a number of items with 
average ratings above 2.0 (moderate relation to learning). The highest rated 
of these suggest that the key to effective instructional design is the flexible 
and appropriate use of a variety of instructional strategies, while maintaining 
an orderly classroom environment. The highest overall rating in this scale was 
for "Use of ... techniques to control classroom disruptiveness." This item was 
followed by "use of prescriptive instruction combined with aspects of informal 
or open education" and "presence of information in the curriculum on 
individual differences and commonalities," both of which explicitly relate to 
student diversity and individualization. Other highly rated items referred to 
specific instructional strategies, including "use of mastery learning techniques, 
... instructional cues, engagement, and corrective feedback, " "use of 
cooperative learning strategies," and "use of diagnostic-prescriptive methods." 

"Curriculum Design" also includes several items with average ratings near 
2.0, although none exceeds the "moderate" level. High ratings were given to 
"materials employ alternative modes of representation" and "degree of structure 



10 



in curriculum accommodates needs of different learners,” both of which 
reinforce the importance of offering a variety of instructional materials and 
approaches to accommodate individual differences. The importance of the 
organization of curriculum content is revealed by two highest-rated items in 
this scale, "materials employ learning hierarchies” and "material is presented in 
a cognitively efficient manner.” 

Implement ation. Classroom Instruction, an d Climate Variables 

This category includes support of the curriculum and the instructional 
program; classroom routines; specific instructional, assessment, and classroom 
management practices; quantity of instruction; academic and nonacademic 
student-teacher interaction; and classroom climate. It is by far the largest of 
the six categories, comprising 79 of the 228 items and eight of the thirty 
scales. Half of these scales had mean ratings above 2.00, placing them among 
the most influential scales overall. 

High ratings in the areas of Implementation, classroom instruction, and 
climate again point up the importance of maintaining an orderly classroom 
environment and providing clear, well organized instruction appropriate to the 
needs of individual learners. In the overall ranking of all 30 scales, 
"Classroom Management” ranked second. Its most critical items were "group 
alerting (teacher uses questioning/recitation strategies that maintain active 
participation by all students)” and "learner accountability (teacher maintains 
student awareness of learning goals and expectations).” Smooth transitions 
from one instructional activity to another, minimal disruptions, and teacher 
awareness of what is going on in the classroom at all times also received mean 
ratings above 2.00. 

"Quantity of Instruction” was ranked third overall, following "Classroom 
Management.” It includes time spent in direct instruction, especially direct 
instruction on basic skills; time spent on homework; and length of the school 
day and the school year. The importance accorded quantity of instruction is 
not surprising. This construct has appeared in many of the most widely cited 
models of school learning (Haertel, Weinstein, & Walberg, 1983). 
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"Student/Teacher Interactions: Social” ranked fourth overall, and 

"Classroom Climate" was ranked fifth. The high ranking for social interactions 
was almost entirely due to just two items with mean ratings of 2.00 or greater, 
"teacher reacts appropriately to correct and incorrect answers," and "student 
responds positively to questions from other students and from teacher."^® 
"Classroom climate" included fifteen items with ratings of 2.00 or greater. 
Taken together, the highly rated items in these two scales characterize a 
classroom in which teacher and students interact considerately and 
cooperatively, where students work with several classmates, share common 
interests and values, and pursue cooperative goals. Students are actively 
engaged in learning, and are involved in making some types of classroom 
decisions. At the same time, the class is well organized and well planned, 
with a clear academic focus. Objectives of learning activities are specific and 
explicit, and students feel continually and appropriately challenged, with the 
pacing of instruction appropriate for the majority. 
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The remaining scales under "Implementation, Classroom Instruction, and 
Climate Variables" have much lower overall ratings, but include more than 
twenty specific items with means of 2.00 or greater. The majority of these 
items refer to instructional organization, and to mechanisms for assuring that 
students understand that organization and the goals of instruction. For 
example, high ratings were given to use of advance organizers and directing 
students’ attention to the content to be learned; and to clear and organized 
direct instruction, systematic sequencing of lesson events, and clear lesson 
transitions. Other highly rated items included corrective feedback in case of 
student error, frequent academic questions, and accurate measurement of skills. 
Finally, the literature strongly supports the teaching of skills in the context of 
meaningful application, use of good examples and analogies, and teaching for 
meaningful understanding, together with explicit promotion of student 
self-monitoring of comprehension and gradual transfer of responsibility for 
learning from the teacher to the student. 





Extramural Variables 



This category includes items associated with the home and community 
contexts within which schools function. As presented above, "Peer Group 
Variables" was ranked sixth among all scales. This was due primarily to the 
emphasis placed on peers’ educational and occupational aspirations, both of 
which had mean ratings of 2.00 or higher. 

Additional highly rated items in this category reflected parental interest 
and involvement in students’ school work. For example, "parental involvement 
in assuring completion of homework," "parental participation in school 
conferences and related activities," and "parental interest in students’ school 
work" all received high ratings. The educational environment of the home 
(e.g., number of books and magazines) was also cited in numerous sources, and 
received consistently high ratings. Student participation in clubs and 
extracurricular school activities and time spent on leisure reading were also 
moderately related to learning outcomes. 

Student Variables 

These are items associated with individual students themselves, including 
demographics, academic history, and various social, cognitive, and affective 
characteristics. Among these items, the one with the highest rating was 
"psychomotor skills specific to area instructed," with a rating of 2.33. This 
was the only item included in the "Psychomotor Variables" scale. However, as 
explained above, this mean is based on only six sources. It is best regarded as 
a statistical artifact, and will not be further discussed. 

"Metacognitive Variables" received the highest mean ratings of any of the 
remaining scales in the entire framework. Highly rated metacognitive items 
include "comprehension monitoring (planning; monitoring effectiveness of 
attempted actions; testing, revising, and evaluating learning strategies)," 
"self-regulatory, self-control strategies (e.g., control of attention)," and 
"positive strategies to facilitate generalization of concepts." 
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A number of specific items in the remaining "Student Variables* scales 
also had high ratings, including positive behavior and ability to make friends 
with peers, motivation for continual learning, and perseverance on learning 
tasks. Highly rated items from the "Cognitive* scale included several 
representing general mental abilities, levels of basic skills sufficient to profit 
from instruction, and prior knowledge in the subject area instructed. 

School Level Variables, and St ate and District Variables 

Educational policy items at the school, district, and state levels appear 
from this research synthesis to have relatively little association with learning 
outcomes, as shown by low mean ratings for categories and scales. A few 
items in this area received mean ratings of 2.00 or higher, but nearly all of 
these were based on fewer than ten sources. Nonetheless, several school-level 
educational practices emerged as important. These included the presence of an 
"effective schools" program; explicit school grading, academic progress, and 
attendance policies; and a safe and orderly school climate. Peer and cross-age 
tutoring, which were classified as school-level variables when their 
implementation required coordination among self-contained classrooms, also 
received moderate or higher ratings based on discussions in more than ten 
sources. 



This research synthesis confirms that distal policy variables are less 
important to schooling outcomes than quantity and quality of instruction, home 
environment, or student characteristics. Of the six categories in the 

conceptual framework (See Tables 1 and 3), "State and District Variables" and 
"School Level Variables," both comprising mainly policy variables, had markedly 
lower mean ratings than the remaining four categories. The items most 
important to learning outcomes are those that are directly tied to students’ 
engagement with the material to be learned. 

In contrast to the earlier view that quality of schooling is of little 
importance relative to out-of-school factors (e.g., Coleman, et al., 1966), this 
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synthesis also suggests that from kindergarten through the twelfth grade, 
across a range of content areas and educational contexts, quality and quantity 
of instruction are roughly equal in importance to student characteristics and 
out'of'school contextual items. 

Furthermore, the present synthesis of educational research is considerably 
more comprehensive than What Works: Research About Teaching and Learning. 
the widely-distributed pamphlet of the U.S. Department of Education (1986); it 
contains both highly effective and relatively less effective practices. The 
present synthesis, moreover, draws on a larger body of literature, and contains 
a more explicit methodology that can be replicated by other investigators. It 
contains some 228 practices in comparison with 41 in the original What Works 
(and 62 in the second edition); and it gives a numerical rating to each one as 
well as composites. Yet none of the findings of What Works and the present 
work are discordant. What Works contains specific findings and elaborates on 
and illustrates various techniques. Such techniques are described specifically 
enough to be understood by parents and teachers; perhaps they might even be 
put into practice without assistance. 

To be useful to practitioners, the present findings, many of which are 
abstract and jncern more complex practices, would have to be further 
described and exemplified. To accomplish this, reviewers would have to return 
to the review literature and perhaps the original studies to analyze the 
specific operational definitions of techniques. These would require translation 
into plain language and prescriptive practices. Many are sufficiently complex 
that they could not be implemented without training and staff development. 
Such an effort would be considerable but worthwhile. 

Turning from the level of the six broad categories to the thirty scales, 
those identified as most important to good learning outcomes are student 
metacognition, effective classroom management, quantity of instruction, positive 
and productive student/teacher Interactions, a classroom climate conducive to 
learning, and a peer culture supportive of academic achievement. These broad 
conclusions are supported by a number of more specific findings from the 
research synthesis. These selected findings are highlighted below. 



Student characteristics 



Individual differences among students have long been recognized as 
critical determinants of learning outcomes, but it was both surprising and 
encouraging that in this synthesis the metacognitive items emerged as most 
important, including comprehension monitoring, use of self-regulatory, 
self-control strategies, and use of strategies to facilitate generalization of 
concepts. Metacognitive variables are heavily cited in the current literature, 
in contrast to an earlier focus on relatively stable general mental abilities. A 
better understanding of these alterable variables may ultimately help the great 
majority of students to reach higher achievement levels through appropriate 
training in metacognition. Two additional student items accorded importance 
in the research literature were "perseverance on learning tasks" and 
"motivation for continual learning." Both of these reinforce the conclusion 
that consistent engagement with the subject matter to be learned is critical to 
school success. 

Quality and quantity of instruction 

Classroom management and climate and student-teacher interactions 
represent an important constellation of variables related to effective 
instruction. Detailed examination of the highly rated items in these areas 
reveals a portrait of cooperative, cohesive, goal-directed classrooms in which a 
variety of educational approaches and activities are employed. Items heavily 
cited in the research literature include sound organization and systematic 
sequencing of instruction, and effective use of direct, teacher-centered 
instruction. Among other instructional approaches frequently linked to positive 
learning outcomes were peer and cross-age tutoring and cooperative group 
learning strategies. 

Several items associated with quantity of instruction also emerged as 
important, including student time on task, length of school day and school 
year, amount of time allocated to direct instruction in basic skills, and time 
spent out of school on homework and on leisure reading. Of these, the most 
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frequently cited variable is time on task. These time-related variables have 
clearly become well established and widely accepted as determinants of learning 
outcomes, in spite of criticisms cited by (Shulman, 1986) of time as an "empty 
vessel." 

Out of school context 

There has been increasing attention in the research literature to the role 
of parental involvement and support variables in promoting student learning. 
The synthesis affirmed the importance of these items, as well as peer group 
influences. These findings were reflected in ratings for parental involvement 
in school activities, interest in schoolwork, and monitoring of school 
attendance and homework completion. Parental support might also be mediated 
through influence on students' selection of friends. Peer group variables, 
especially academic and occupational aspirations, were found to be strongly 
related to school success. 

Strength of influences on school learning 

Physical processes can often be explained as functions of a small number 
of variables interacting in simple ways. In contrast, schooling processes 
respond to a multitude of influences interacting in kaleidoscopic patterns. 
This research synthesis has confirmed that a large number of variables are 
moderately related to learning outcomes, but few if any single variables are 
very strongly related to learning. Authors of original research studies and of 
reviews and syntheses are appropriately cautious in stating the importance of 
particular items, and their caution is reflected in the relatively narrow range 
of mean ratings shown in Table 3. Nonetheless, taken together, the items 
examined in this synthesis are powerful determinants of school effects. 

The conclusions discussed in this section are based on what appears to be 
the most comprehensive analysis of the literature on effective educational 
practices for regular and special education. Related work involving the 
consensus of the panel of experts, the authors of research reviews, and regular 
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and special practitioners adds further support to the conclusions (Wang, 
Walberg, & Reynolds, 1989; Reynolds, Wang, & Walberg, 1989). 
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Still, certain caveats should be noted: It cannot be determined from the 

analyses, for example, what actual effect sizes will result; the analyses merely 
estimate their relative sizes. In addition, the analyses yield neither actual nor 
relative estimates of combinations of practices. It would seem reasonable to 
suppose that implementation of more practices with the highest estimates would 
yield the largest effects, but this supposition is a matter for subsequent 
empirical research. 

Another caveat applies to the content analysis of research literature on 
group-level effects, notably the literature on effective schools. Some of the 
effective schools factors have been analyzed in relation to school averages on 
achievement tests. Such relationships might be found somewhat larger or 
smaller if calculated for individual children. It can be expected that expert 
reviewers on this subject (on which the syntheses depend) would take this 
uncertainty into consideration in interpreting their findings. It has rarely 
been demonstrated that, techniques that work for the average student have 
deleterious consequences for other students’ learning. 

Nonetheless, it is worth keeping this limitation in mind in interpreting 
the findings and in tracing their implications. There are many other cautions 
that ordinarily apply to educational research, such as the possibility that 
effective methods found a decade ago no longer apply today. These are 
obvious enough to leave to researchers and experienced educators as they 
think about how the findings apply in their own situations. 
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NOTES 



1. This research was supported in part by the Temple University Center for 
Research in Human Development and Education, and in part by a grant from 
the U. S. Department of Education’s Office of Special Education and 
Rehabilitative Services. The opinions expressed herein are solely those of the 
authors, and no official endorsement should be inferred. 

2. Copies of the detailed coding form and complete bibliographic citations for 
the 179 sources, as well as copies of the data archive are available from Dr. 
Margaret C. Wang at the Center for Research in Human Development and 
Education, Ritter Hall Annex, 9th Floor, Temple University, Philadelphia, PA 
19122. 

3. In addition to the coding and analysis of the 179 source documents, a 
survey was also conducted of the authors of all major source documents 
examined. The summary coding form described below was distributed to 
authors, with a request to provide overall ratings of the importance of the 228 
items to learning outcomes. A total of 78 forms were returned. These expert 
ratings were analyzed separately from the source document ratings, following 
identical procedures. Results were highly similar, with the exception that the 
experts generally tended to give somewhat higher numerical ratings. 

4. This panel included 12 prominent experts in areas of research on teaching, 
education, educational psychology, and special education. 

5. Most of these occurred for items in the scale, "History of Educational 
Placements," which accounts for the low mean of this variable in Table 3. 

6. If any supplementary items had been coded, these were reexamined as the 
forms were transcribed, and whenever possible were included under one of the 
prespecified items. This was generally possible because most supplementary 
items documented authors’ more detailed or specific empirical conclusions, for 
example, specific types of motivation related to learning, or particular variants 
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of instructional practices. Such detailed findings were incorporated into the 
broader variable prespecified on the form. The other major group of 
supplementary items were those documenting two*way or occasionally 
higher-order interactions. Because interactions represent more subtle findings 
and frequently fail to replicate, they were not transferred from the detailed 
form to the summary form. The summary forms were keyed and verified, and 
files were prepared for data analysis using standard statistical software 
packages. 

7. The weights used were equal to the numbers of original items included in 
the respective scales. Note that if there were no missing data, this procedure 
would result in giving all of the original items in a broad category equal 
weight. Where some items in a scale are missing, this procedure in effect 
assigns the mean of the nonmissing scale items to those missing observations. 
For any given scale, about 15 percent of the values of items on average were 
missing. 

8. As noted in the footnote to the table, these reported reliabilities are for 
means of all the items in a given category or scale. Due to missing data, 
values for some sources were based on means of fewer items. 

9. The highest ratings overall were assigned to "Psychomotor Variables," and a 
moderately high rating was also assigned to the scale "Accessibility Variables." 
However, only one item was included in each of these scales, and these items 
were referred to in six or fewer of the 179 sources analyzed. Thus, 
"Psychomotor Variables" and "Accessibility Variables" were set aside. The list 
of scales with the highest ratings include the 28 scales with more items and 
more ratings. 

10. A third item in this scale, "teacher provides explicit coaching to reduce 
aggression," also received a mean rating above 2.00, but was mentioned in only 
4 of the 179 sources. This item is of limited relevance in most regular 
educational settings. 
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Table 1 



Conceptual Framework with Illustrative Examples 



Category/Subcategory* 



Illustrative Variable 



Category I. State and District Variables : These are variables associated with 
state and district level school governance and administration. They include 
state curriculum and textbook policies, testing and graduation requirements, 
and teacher licensure; as well as specific provisions in teacher contracts, and 
some district-level administrative and fiscal variables. 

District Level Demographics and School district size 

Marker Variables 

State Level Policy Variables Teacher licensure requirements 



Category TT. Out of School Contextual Variables : These are variables 
associated with the home and community contexts within which schools 
function. They include community demographics, peer culture, parental support 
and involvement, and amount of time students spend out-of-school on such 
activities as television viewing, leisure reading, and homework. 



Community Variables 



Peer Group Variables 



Home Environment and Parental 
Support Variables 

Student Use of Out of School Time 
Variables 



Socioeconomic level of 
community 

Level of peers’ academic 
aspirations 

Parental involvement in 
assuring completion of 
homework 

Student participation in clubs 
and extracurricular school 
activities 



Category 111. School Level Variables : These are variables associated with 
school-level demographics, culture, climate, policies, and practices. They 
include demographics of the student body, whether the school is public or 
private, and levels of funding for specific categorical programs; school-level 
decision making variables, and specific school-level policies and practices, 
including policies on parental involvement in the school. 

Demographic and Marker Variables Size of school 

Teacher/ Administrator Decision 
Making Variables 
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Principal actively concerned 
with instructional program 



Table 1 (Category III, continued) 



Category/Subcategory 


Illustrative Variable 


School Culture Variables 

(Ethos Conducive to Teaching and 

Learning) 


School-wide emphasis on and 
recognition of academic 
achievement 


School-Wide Policy and Organizational 
Variables 


Explicit school-wide discipline 
policy 


Accessibility Variables 


Accessibility of educational 
program (overcoming 
architectural, communication, 
and environmental barriers) 


Parental Involvement Policy 
Variables 


Parental involvement in 
improvement and operation 
of instructional programs 



Category IV: Student Variables: These are variables associated with individual 
students themselves, including demographics, academic history, and a variety of 
social, behavioral, motivational, cognitive, and affective characteristics. 



Demographic and Marker Variables 


Gender 


History of Educational Placement 


Prior grade retentions 


Social and Behavioral Variables 


Positive, nondisruptive behavior 


Motivational and Affective Variables 


Attitude toward subject matter 
instructed 


Cognitive Variables 


Level of specific academic 
knowledge in subject area 
instructed 


Metacognitive Variables 


Comprehension monitoring 
(planning: monitoring effective- 
ness of attempted actions; 
monitoring outcomes of actions; 
testing, revising, and evaluating 
learning strategies) 


Psychomotor Variables 


Psychomotor skills specific to 
area instructed 



Table 1 (continued) 



Category/Subcategory 



Illustrative Variable 



Category V. Program Design Variables : These are variables associated with 
instruction as designed, and with the physical arrangements for its delivery. 
They include the instructional strategies specified by the curriculum, and 
characteristics of instructional materials. 



Demographic and Marker Variables 



Curriculum and Instructional 
Variables 

Curriculum Design Variables 



Size of instructional group 
(whole class, small group, 
one-on-one instruction) 

Alignment among goals, 
contents, instruction, 
assignments, and evaluation 

Materials employ advance 
organizers 



Category VI. Implementatinn. Classroom Instruction, and Climate Variables : 
These are variables associated with the implementation of the curriculum and 
the instructional program. They include classroom routines and practices, 
characteristics of instruction as delivered, classroom management, monitoring 
of student progress, and quality and quantity of instruction provided, as well 
as student-teacher interactions and classroom climate. 



Classroom Implementation Support 
Variables 



Classroom Instructional 
Variables 

Quantity of Instruction 
Variables 



Classroom Assessment 
Variables 



Classroom Management 
Variables 



Establishing efficient 
classroom routines and 
communicating rules and 
procedures 

Use of clear and organized 
direct instruction 

Time on task (amount of time 
students are actively engaged in 
learning) 

Use of assessment as a frequent, 
integral component of in- 
struction 

Group alerting (teacher uses 
questioning/recitation strategies 
that maintain active partici- 
pation by all students) 



Table 1 (Category VI, continued) 



Category/Subcategory 



Illustrative Variable 



Student and Teacher Interactions: 
Social Variables 



Student and Teacher Interactions: 
Academic Variables 



Classroom Climate 
Variables 



Student responds positively to 
questions from other students 
and from teacher 

Frequent calls for extended, 
substantive oral and written 
response (not one-word answers) 

Cohesiveness (members of class 
are friends sharing common 
interests and values and 
emphasizing cooperative goals) 



^ubcategories are listed below the description of each broad category, and are 
each illustrated with representative variables. For example, the first broad 
category includes two subcategories, ’District Level Demographics and Marker 
Variables," and "State Level Policy Variables.* 



Table 2 



Number and Percent of Source Documents by Type 



Type of Source® 


N 


Percent 


Total Pages 


Chapters from Annual Review Series 


86 


48 


3,179 


Handbook Chapters 


44 


25 


1,089 


Government Documents and 
Commissioned Reports 


20 


11 


772 


Book Chapters 


18 


10 


563 


Review Articles in Journals 


11 


6 


152 


Total 


179 


100 


5,755 



complete bibliography is available from the first author 
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Table 3 



Reliabilities, means, standard deviations, and frequencies for source ratings 



Category/Subcategory Reliability* Mean S.D. Frequency 



State and District Variables 

District demographics & marker vars. 
State level policy variables 

Out of School Contextual Variables 
Community variables 
Peer group variables 
Home environment & parental support 
Student use of out-of-school time 

School Level Variables 

Demographics and marker variables 
Teacher/administrator decision making 
School culture variables 
School-wide policies and organization 
Accessibility variables 
Parental involvement policy variables 

Student Variables 

Demographics and marker variables 
History of educational placements 
Social and behavioral variables 
Motivational and affective variables 
Cognitive variables 
Metacognitivie variables 
Psychomotor variables 

Program Design Variables 

Demographic and marker variables 
Curriculum and instruction variables 
Curriculum design variables 

Classroom Instruction and Climate V ariables 
Classroom implementation support 
Classroom instructional variables 
Quantity of instruction variables 
Classroom assessment variables 
Classroom management variables 
Student/teacher interactions: Social 
Student/teacher interactions: Academic 
Classroom climate variables 



.90 


1.22 


.81 


27 


.95 


1.46 


.50 


14 


N.C. 


1.24 


1.00 


19 


.99 


1.87 


.39 


59 


N.C. 


1.80 


.41 


15 


.98 


2.00 


.34 


18 


.95 


1.90 


.40 


47 


N.C. 


1.94 


.46 


17 


.95 


1.54 


.96 


102 


.91 


1.74 


.56 


25 


.87 


1.65 


.95 


21 


.87 


1.84 


.43 


49 


.76 


1.40 


1.14 


74 


N.C. 


2.00 


.00 


2 


N.C. 


1.67 


.56 


23 


.92 


1.83 


.57 


155 


.71 


1.70 


.77 


90 


N.C. 


0.16 


1.80 


19 


.80 


1.98 


.34 


35 


.91 


1.93 


.42 


81 


.88 


1.98 


.33 


101 


.91 


2.08 


.36 


76 


N.C. 


2.33 


.52 


6 


.90 


1.90 


.38 


142 


N.C. 


1.97 


.54 


23 


.90 


1.92 


.46 


108 


.89 


1.88 


.34 


97 


.97 


1.84 


.66 


165 


.85 


1.84 


.38 


66 


.89 


1.85 


.74 


156 


.94 


2.02 


.64 


69 


N.C. 


1.89 


.30 


61 


.98 


2.07 


.23 


42 


.73 


2.02 


.41 


44 


.77 


1.89 


.44 


29 


.99 


2.01 


.38 


75 



*Coefficient alpha reliabilities were estimated for each scale from average variances and 
inter-item covariances. Due to missing data, ratings for some cases are based on fewer 
items. Thus, obtained reliabilities are somewhat lower than the figures reported in this 
table. "N.C.* indicates values that were not calculable, either because scales, consisted of 
only a single item, or due to patterns of missing data. 
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